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I.  SUMMARY 


Under  contract  to  the  Air  Force  Materials  Laboratory,  Wright  - 
Patterson  Air  Force  Base  (W-PAFB),  Ohio,  the  University  of  Dayton 
has  indexed  and  processed  some  30,000  documents  for  the  Aerospace 
Materials  Information  Center.  Index  terms  have  been  added  to  computer 
tapes  for  retrieval  and  printing  of  the  keyword  index. 

The  final  choice  of  an  information  retrieval  system  was  an  in¬ 
verted  file  system  using  deep  indexing,  links,  and  roles;  all  material 
was  processed  on  the  University  NCR  304  computer.  The  initial  pur¬ 
pose  of  the  computer  operations  was  the  the  tabulation  of  a  thesaurus 
of  the  system's  vocabulary  and  the  generation  of  a  keyword  index  that 
could  be  used  by  individuals  in  performing  manual  literature  searches. 
The  magnetic  tape  from  which  the  thesaurus  and  dictionary  were  gen¬ 
erated  also  provided  the  opportunity  to  perform  mechanized  searches 
as  the  need  arose. 

Within  the  past  year,  all  data  in  the  system  have  been  added  in 
sequential  format  and  without  roles  to  the  IBM  7094  at  W-PAFB,  which 
is  now  used  for  searching,  though  the  capability  still  exists  at  the 


-1 


University  of  Dayton.  In  addition  to  searches,  the  IBM  7094  produces 
the  KWIC  index  section  of  the  "Materials  Information  Bulletin.  " 

Currently  all  documentation  received  at,  or  issued  by,  the 
Ceramics  &  Graphites  Technical  Information  Center  is  included  in  the 
overall  program  developed  and  operated  by  the  University  of  Dalton. 

All  of  the  30,000  documents  on  file  at  the  Aerospace  Materials 
Information  Center  are  unclassified.  The  collection  is  growing  at  the 
rate  of  400  per  month.  Reference  use  of  .  ie  collection  and  abstracts 
has  averaged  about  300  per  month,  with  loans  (approximately  600  to  700 
annually  beside  use  in  the  library)  being  made  to  Materials  Laboratory 
personnel  and  in  a  few  instances  to  people  located  outside  Wright  - 
Patterson  AFB. 

A  weekly  "Materials  Information  Bulletin”  announces  to  potential 
users  all  new  additions  to  the  collection;  the  "Abstracts  of  Active  Con¬ 
tracts"  announces  current  projects  of  the  Laboratories.  Bibliographies 
are  prepared  as  requested.  The  majority  of  requests  are  by  telephone, 
though  requests  by  memo  or  letter  are  received  quite  frequently.  Fifty 
percent  of  all  requests  are  made  by  DoD  contractors;  the  remaining 
50  percent  cover  requests  from  various  government  activities. 
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II.  MECHANIZATION 


1.  CHRONOLOGY 

In  early  I960,  the  Materials  Information  Branch  was  established 
as  a  centralized  technical  information  service  for  the  Air  Force  Mater¬ 
ials  Laboratory.  A  processing  group  was  organized  to  assist  in. the 
problem  of  storage,  retrieval,  and  use  of  information  dealing  with  the 
materials  R&D  area.  From  this  original  group,  the  Air  Force  Mater¬ 
ials  Information  Centers  (AFMIC)  have  evolved.  The  Aerospace  Ma¬ 
terials  Information  Center  and  the  Ceramics  and  Graphites  Technical 
Information  Center  are  two  of  the  eight  information  centers.  (Appen¬ 
dix  A  illustrates  the  eight  centers.) 

Contract  with  the  University  of  Dayton  was  let  December  1,  1960, 

i 

to  perform  indexing  and  develop  a  computer-assisted  information  sys¬ 
tem. 

Actual  work  on  the  contract  with  the  University  of  Dayton  began 
in  September  1961.  The  system  was  developed  and  input  began. 
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In  mid- 1963,  the  retrieval  program  became  operational.  The 
program  until  this  time  consisted  only  of  input.  The  thesaurus  was 
revised  to  retain  as  much  specificity  as  practical  while  using  a  frag¬ 
mentation  system  for  organic  chemicals  and  applying  a  modified  but 
similar  approach  to  other  areas  where  an  uncontrolled  increase  in 
vocabulary  appeared  detrimental.  The  established  controls  have  pro¬ 
vided  a  fairly  stable  vocabulary  and  eliminated  the  need  for  continued 
revision  of  the  computer  tapes. 

In  1964,  the  retrieval  program  was  rewritten  for  the  IBM  7094  at 
W-PAFB.  In  this  transition,  the  inverted  file  was  changed  to  sequen¬ 
tial.  The  data  base  was  added  in  September  although  the  University  of 
Dayton  still  formulates  the  search  questions.  Thesaurus  maintenance, 
Keyword  index  printing,  and  data-base  updating  are  still  being  per- 
'ormed  on  the  NCR  304  at  the  University  of  Dayton. 

After  a  year  of  setting  up  the  program  on  the  IBM  7094,  the  re- 
:rieval  program  became  operational  in  late  1965. 

>.  MAJOR  PROBLEMS  IN  DEVELOPMENT 

Problems  with  the  free  vocabulary  began  developing  within  a 
:ouple  of  years,  initiating  a  redevelopment  of  terms  within  the  system 
md  the  establishment  of  vocabulary  controls. 
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Indexing  is  usually  a  chore,  although  some  relief  is  given  when 
the  indexer  formulates  search  questions  as  well  as  indexing.  Plans 
call  for  indexers  to  supervise  students  in  the  indexing  and  spend  part  of 
their  time  doing  more  interesting  and  varied  work. 

The  University's  experience  has  shown  negative  aspects  to  the 
system  such  as  the  difficulty  encountered  in  applying  role  indicators  to 
the  terms.  After  a  thorough  evaluation,  the  assigning  of  role  indicators 
was  dropped.  It  was  also  discovered  that  a  large  amount  of  time  could 
be  saved  in  the  editing  operation  by  letting  the  computer,  rather  than 
the  editors,  do  the  cross- relating  of  terms. 

Another  problem  developed  is  the  indexing  of  progress  reports, 
which  increased  the  number  of  overall  entries  to  the  system.  This  pro¬ 
cedure  was  changed;  currently,  if  a  progress  report  is  received  in  the 
Center  and  it  is  the  first  to  be  received  on  the  contract,  it  is  assigned 
a  new  accession  number  and  processed  for  indexing.  If  it  is  not  the 
first,  the  document  is  assigned  the  same  accession  number  affixed  to 
the  first  report  and  filed.  Because  of  this  method  of  handling  progress 
reports,  the  exact  size  of  the  system  is  indeterminate. 
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THESAURUS  DEVELOPMENT 


The  use  of  deep  indexing  with  links  and  roles  formulated  the  basis 
of  this  system.  The  role  indicators  used  were  a  slightly  modified  ver¬ 
sion  of  those  used  by  the  DuPont  Company.  The  first  500  documents 
that  were  indexed  averaged  over  80  link-to- role-term  combinations  per 
report.  As  the  information  analysts  gained  experience,  the  average 
number  of  entries  per  document  was  reduced  to  slightly  over  40.  The 
elimination  of  roles  from  the  system  reduced  the  average  to  about  27 
entries  per  document. 

In  1963,  when  controls  were  established  for  the  naming  of  ma¬ 
terials,  the  18,000  items  in  the  system  were  reduced  to  about  10,000. 
Within  the  past  year,  some  6,000  documents  being  added  to  the  system 
have  generated  very  few  new  terms. 

Twelve  to  thirteen  terms  are  assigned  for  each  document,  but 
because  of  the  hierarchial  relationships,  a  document  may  then  be  posted 
to  about  20  to  25  terms  automatically. 

Maintenance  of  the  thesaurus  is  described  in  Appendix  B. 


DESCRIPTION  OF  PROCESSES 


Until  I960,  a  collection  of  Materials  Laboratory's  and  its  con¬ 
tractors'  reports  had  been  maintained  manually.  Reports  were  logged 
in,  and  a  report  number  index  card  maintained.  No  subject  cross- 
referencing  was  available. 

Since  the  beginning  of  the  University  of  Dayton  contract,  material 
added  to  the  computer  also  has  had  abstract  cards  prepared  for  it. 
Currently,  these  abstract  cards  are  maintained  at  the  Materials  Labor¬ 
atory  by  contractor,  contract  number,  agency,  title,  and  accession 
number.  Card  sets  are  also  maintained  at  the  University  of  Dayton  and 
some  of  the  divisions  within  the  Laboratory. 

The  computer  programs  developed  and  operated  by  the  University 
of  Dayton  permit  computer  searches  and  also  produce  a  thesaurus  and 
a  printed  index  of  the  entire  collection  (see  Figure  1  for  input  flow). 
Searches  and  printing  of  the  KWIC  index  are  performed  from  the  same 
data  base  using  the  IBM  7094  at  the  Materials  Laboratory.  In  order  to 
provide  information  required  to  produce  the  outputs  just  mentioned,  the 
following  procedures  are  followed: 


-7- 


Input  Flow  of  Information 


(1)  Input  Procedures 


1.  Document  is  received,  checked  for  duplication,  and 
reviewed  for  pertinency  to  the  collection. 

2.  Accession  numbers  (six  digits)  are  assigned  by  the 
University  of  Dayton  staff  located  at  the  Materials  Labora¬ 
tory. 

3.  Preliminary  record  card  is  typed  showing  accession 
number,  title,  date,  and  contractor. 

4.  Documents  are  forwarded  to  the  University  of  Dayton 
for  indexing  and  processing. 

5.  Final  abstract  card  is  typed  including  complete  bibli¬ 
ographic  information. 

6.  Indexer  indexes  the  document  using  an  indexing  work¬ 
sheet  (referred  to  as  index  card;  see  Figure  2).  This  index 
card  includes  the  accession  number  (access  number),  title 
of  the  report,  author  or  authors,  indexer's  name  and  date, 
and  the  terms  and  appropriate  links  assigned.  A  link  is  a 
unique  alphabetic  suffix  added  to  the  accession  number. 
Terms  are  selected  from  the  thesaurus  by  following  cer¬ 
tain  ground  rules.  See  Appendix  C  for  a  further  explanation 
of  these  rules. 
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FIGURE  2 
Index  Card 
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7.  Document  and  index  card  go  to  the  review  indexer  for 
pre-editing. 

8.  Indexed  documents  are  returned  to  the  Materials  Lab¬ 
oratory. 

9.  Index  and  vocabulary  cards  are  keypunched. 

10.  Cards  are  processed  on  the  computer;  terms  not  al¬ 
ready  on  tape  are  rejected. 

11.  Reject  ;d  terms  are  sent  to  editors  for  review.  Terms 
are  held  for  further  review  or  added,  if  usage  has  shown 
need  for  the  term.  Modifications  to  the  various  sections  of 
the  valid  vocabulary  are  made  as  described  in  Appendix  B. 
Figure  3  shows  the  various  card  formats. 

12.  Final  abstract  cards  are  returned  to  the  Materials 
Laboratory  and  filed. 

13.  Final  record  cards  are  filed  by  the  University  of 
Dayton  staff,  including  abstract  and  index  cards. 

(2)  Outputs 


1 .  Thesaurus 

The  thesaurus  contains  some  10,000  terms,  printed 
in  numeric  sequence  by  numeric  code  (see  Appendix  D-l 
for  sample).  Generic  Terms  (GT),  Related  Terms  (RT), 


-11- 


0  <«  a  x 

A  a  a  a 

ft  ft  ft  « 
«-l  H 

<  <  <  3 


o 

00 

i 

H 

CM 


w  u 

CO  CO 

£  £ 

•  o 

wo  .o 

5*  >* 

SC  £ 


0  0 
A  A 
P.  P. 
H  H 


O  O 
•rl  O 
X  X  U  O 

S  E  ©  o 

a  0  e  o 

ss  a  i  8 


Ri 

i 

H 


oo 

H 

I 


o  u  u  o 

•fi  H  -rl  -rl 

P  P  P  P 

0  ©  ®  4) 

ESSE 

2  5  3  3 

2  2  55  2 


I 

o\ 


P  J< 
«  c 
e  0 

£  ?! 


eh 

« 

r“% 

*  O 
e-l  *rl 
C  P 

0 

•  E 

£ 


o 

cm 

i 


p 

0 

s 

3 

35 


P 

0 

S 

3 

SS 


0 

Sh 

0 

H 

P 

A 

0 

0 

A 

O 

B 

s, 


0 

A 

C. 


cs 

cm 

i 

fvl 


0 

A 

P. 


2 


h  o 

«3  O 


o 

H 

I 

-a- 


o  o 
•H  *rt 
P  P 
0  0 
B  B 
3  3 

SS  ;s 


CO 

ON 

H 

P  0 

0  U 

ON 

H 

B 

1 

a  o 

1 

1 

•H 

CM 

3  < 

<\j 

CH 

H 

H 

2  w 

rt 

JK 

B 

0 

a 


_  .* 
0  S 
A  -rl 
P.H 

<  •» 

•»  =it 

U  0 
-rl  0 


C- 

CM 

I 

H 

CM 


P 

0 

a 

3 

ss 


0-g 

x:  -r! 
p.  H 

iH 

•<  «a 
•a  =* 

O  0 
•H  0 
P  0 
0  O 
B  O 
3  H 
SB  w 


P  O  P.  (P 


CO 

s> 

K 

P 

«< 

CO 

w 

K 

Hi 


I 


M 

o  o  a. 

•H  -H  < 

P  P  Eh 

0  0 

B  B  U 

3  3  M 

35  SS  tt 

U 
35 
CO 
C 


Hi  <  P 


X 

w 


o 

2 

M 

Eh 

CO 

2 


CO 

2 

O 


P 

P 

< 


p 

10 
B  CO 

3  2 

2  O 

M 

m 

* 

p 


I 


P 

0 

B 

3 

2 


CO 

P. 

M 

P 

pH 


M 


CO 

to 

w 

o 

u 

•< 


p 

0 

B 

3 

2 


u 


-12- 


FIGURE  3 
arc!  Formats 


and  Posted  On  (PO)  terms  are  listed  immediately  below  the 
main  term  entries  with  the  code  GT,  RT,  or  PO.  "See" 
terms  and  "see  also"  (SA)  terms  are  included  in  the  the¬ 
saurus  . 

2.  Keyword  Index 

Published  annually,  with  supplements  as  4,000  or 
5,000  entries  are  accumulated,  this  index  (Appendix  D-2) 
provides  a  manual  searching  tool  of  all  items  in  the  collec¬ 
tion.  Each  term  is  shown  with  all  the  accession  numbers 
posted  to  it.  The  accession  numbers  may  then  be  used  to 
locate  the  abstract  card  with  full  bibliographic  data  or  the 
document  itself.  To  simplify  this  manual  search,  accession 
numbers  are  printed  in  10  columns  corresponding  to  the 
terminal  digit  of  the  number.  The  index  is  kept  at  the  lab¬ 
oratory  for  quick  ready  reference -type  searches  not  requir¬ 
ing  computer  processing. 

The  index  as  well  as  the  thesaurus  is  produced  on  the 
NCR  304  at  the  University  of  Dayton.  Frequency  of  use  can 
be  determined  from  the  index  printout  but  not  from  the 
thesaurus. 
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3. 


Document  Search 


Search  requests  are  received  by  the  Materials  Labo¬ 
ratory  and  sent  to  the  University  of  Dayton  for  processing. 
The  search  writer  formulates  the  question  and  writes  up  the 
search  in  English  (Figure  4).  The  word  numbers  are  usu¬ 
ally  looked  up  by  one  of  the  student  employees.  The  codes 


IR  WORKSHEET  FOR  SEARCH  # _ 

PAGE— .OF 


WORD 

NUMBER 


FIGURE  4 


CONNECTOR 


AND  OR  NOT 


WORD 


are  transferred  to  a  keypunch  form  (Figure  5)  used  in  punch¬ 
ing  cards  for  the  retrieval  program  on  the  IBM  7094  at  the 
Materials  Laboratory. 


INFORMATION  RETRIEVAL  -  SEARCH  INPUT 
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FIGURE  5 


Resultant  printouts  (Appendix  D-3)  indicate  the  search 
number,  search  title,  requester's  name  and  extension,  spe¬ 
cial  instructions  on  the  search,  order  of  terms,  corrections, 
term  numbers,  and  terms  to  be  searched.  Results  are  in 
the  form  of  accession  numbers  for  documents  containing  all 
or  some  of  the  terms  specified  in  the  search  question.  Ac¬ 
cession  numbers  are  printed  in  10  columns. 

Printouts  are  reviewed  by  analysts  at  the  Center  be¬ 
fore  the  requester  receives  a  copy.  The  analyst  may  indi¬ 
cate  items  which  seem  to  be  best  suited  to  the  requester's 
needs,  ones  which  look  doubtful,  or  ones  which  are  ruled 
out  immediately.  Local  users  will  receive  a  printout  of 
terms;  outside  requesters  will  receive  a  list  of  references 
or  the  actual  information  they  required. 

4.  KWIC  Index  to  "Materials  Information  Bulletin" 

Weekly,  the  IBM  7094  prints  a  KWIC  Index  of  new  doc¬ 
uments  added  to  the  collection  (See  Appendix  D-4  for  sam¬ 
ple).  Permuted  titles  refer  to  the  accession  number  (found 
at  the  right-hand  side  of  the  page  for  each  title).  This  ac¬ 
cession  number  may  be  used  to  locate  either  the  abstract 
or  document.  A  request  form  (added  two  months  ago)  is 
provided  at  the  back  of  the  bulletin,  allowing  the  user  to 
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circle  any  accession  numbers  pertaining  to  documents  for 
which  he  wishes  to  see  an  abstract.  This  form  is  returned 
to  the  Materials  Laboratory,  and  abstracts  are  furnished 
as  soon  as  possible. 

5.  Other 

A  number  of  printouts  are  made  to  assist  in  system 
management.  The  vocabulary  may  be  printed  in  alphabetic 
or  numeric  sequence.  The  Generic  File  may  be  printed, 
creating  a  listing  in  the  same  format  as  the  thesaurus,  dif¬ 
fering  only  in  that  all  terms  at  all  levels  appear.  Also, 
lists  of  errors  are  printed  by  each  of  the  updating  runs  for 
correction. 

5.  ACTIVITIES  BEING  PLANNED  OR  DEVELOPED 

FOR  MECHANIZATION 

It  is  hoped  that  the  "Abstracts  of  Active  Contracts,"  now  being 
produced  manually,  can  be  printed  from  the  DD  Form  1498  file  at  DDC, 
directly  from  the  computer  to  reproducible  mats. 

A  Miracode  system  just  purchased  will  be  used  to  store  Materials 
Laboratory  information  as  well  as  the  "Commerce  Business  Daily" 
entries  and  some  AEC  materials  information  being  purchased  on  film 
from  the  AEC. 
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Microfilm  will  be  obtained  on  all  other  older  documents,  micro 
fiche  on  the  more  current. 
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III.  PROGRAM  SYSTEM  DATA 


1.  FILES 

(1)  Vocabulary  File  (Master  Word  List) 

This  is  the  basic  file  of  valid  descriptor  terms  and  their 
numeric  code  equivalents  and  also  contains  "see"  terms.  Two 
versions  of  the  file  are  kept,  one  in  numeric  sequence  which  is 
nearly  alphabetic  because  of  the  code  assignment  technique,  and 
another  in  computer  alphabetic  sequence  which  is  a  slightly  dif¬ 
ferent  sequence.  These  and  the  Thesaurus  File  are  the  only  tapes 
in  the  system  that  contain  English. 

(2)  Generic  File 

This  file  contains  records  that  reflect  the  hierarchical  re¬ 
lationships  of  the  vocabulary.  It  consists  of  pairs  of  term  code 
numbers,  the  first  being  posted  on  or  generically  lower  than  the 
second. 
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(3)  Thesaurus  File 


This  is  the  complete  dictionary  for  the  system.  The  the¬ 
saurus  is  not  required  by  the  computer  system  except  as  an  in¬ 
dexing  aid  and  a  search  writing  aid.  It  contains  the  codes,  first- 
order  generic  terms  (both  higher  and  lower),  "see"  terms,  "see 
also"  terms,  and  related  terms.  Bombardment  and  irradiation 
are  examples  of  "see  also"  terms;  each  is  a  good  term  but  there 
is  overlap  in  meaning. 

(4)  Posting  Index  (Search  File) 

This  inverted  index  contains  term  code  and  accession  num¬ 
ber  pairs  in  32-word  records.  When  it  is  transferred  to  the  7094 
system,  it  is  reinverted  to  document  sequence. 

2.  PROGRAMS 

( 1)  Input 

Data  cards  containing  an  English  descriptor  and  the  docu¬ 
ment  accession  number  are  read.  The  English  descriptor  is 
compared  to  the  Vocabulary  File.  If  the  term  is  valid,  the  code 
number  and  accession  number  pair  is  written  on  a  work  tape. 
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The  Generic  File  is  examined,  and  all  higher  order  terms  are 
selected  for  automatic  posting.  These  are  also  written  on  the 
work  tape.  Contents  of  invalid  cards  are  printed  for  reassess¬ 
ment. 

(2)  Post 

The  work  tape  resulting  from  the  Input  program  is  sorted 
by  code  numbers  and  posted  in  proper  sequence  in  the  Posting 
Index. 

(3)  Search 

Batches  of  four  to  six  questions  (maximum  of  92  words  per 
question)  from  one  or  more  users  are  read  and  stored.  The  term 
codes  of  the  questions  are  compared  to  the  Posting  Index,  and 
their  Boolean  relationships  are  evaluated.  Evaluation  takes  place 
serially  from  first  term  to  last  and  on  two  levels.  AND  or  NOT 
terms  are  on  the  higher  level  and  OR  terms  are  on  the  lower 
level.  Thus  the  truth  of  a  single  OR  term  in  a  string  makes  true 
that  section  of  the  AND  statement  in  which  it  appears. 

Provision  is  made  for  limiting  the  total  number  of  document 
retrievals  desired  and  for  selecting  the  number  of  true  phrases 
to  constitute  a  hit. 
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Accession  numbers  of  retrieval  documents  are  printed  in 
numeric  order  within  groups  showing  the  number  of  hit  terms. 

Two  versions  of  this  program  have  been  written,  one  for 
the  NCR  304  and  one  for  the  IBM  7094. 

(4)  Print  Thesaurus 

The  preparation  of  the  Print  Thesaurus  tape  involves  match- 
:ng  input  time  with  the  Master  Word  List  tape.  A  special  sub¬ 
routine  is  required  to  add  new  terms  to  these  tapes.  A  cross¬ 
match  is  made  to  establish  that  the  same  terms  are  included  on 
both  tapes.  The  Print  Thesaurus  tape  is  used  to  produce  multiple 
copies  of  the  thesaurus  off-line. 

The  thesaurus  is  printed  in  numeric  sequence.  All  first- 

order  generic  terms,  related  terms,  "see"  terms,  and  "see  also" 

1 

terms  are  printed. 

(5)  Print  Keyword  Index 

English  terms  are  extracted  from  the  Vocabulary  File  and 
merged  with  data  from  the  Posting  Index  onto  a  work  tape.  The 
tape  is  sorted  into  sequence  by  the  English  term  and  accession 
numbers,  and  printed  off-line.  A  supplement  to  the  Posting  Index 
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is  printed  when  4,000  or  5,000  entries  are  accumulated.  Once 
each  year,  all  the  supplements  are  merged  and  a  complete  ver¬ 
sion  is  printed. 

(6)  Print  Vocabulary- Numeric 
Print  Vocabulary- Alphabetic 

The  Vocabulary  file  of  codes  and  English  is  printed  in  the 
sequence  indicated.  To  date,  the  alphabetic  sequence  has  not 
been  printed. 

(7)  Print  Generic 

The  Generic  File  is  printed  in  numeric  sequence. 

(8)  Vocabulary  Update 

This  program  makes  additions  and  deletions  to  the  Vocab¬ 
ulary  File.  An  addition  is  a  new  term  and  an  unused  code  number. 
Code  numbers  are  assigned  so  that  a  nearly  alphabetic  sequence  is 
maintained.  Unnecessary  terms  are  deleted  from  the  file.  Those 
terms  are  also  deleted  from  the  Posting  Index. 

(9)  Flip 

This  program  performs  interchange  of  one  term  for  another. 
It  deletes  the  first  term  from  the  Vocabulary  File  and  inserts  the 
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second.  It  then  moves  the  postings  from  the  first  to  this  second 
term  in  the  Posting  Index.  If  the  second  term  number  is  zero, 
all  entries  in  the  Posting  Index  are  simply  deleted. 

(10)  Change 

The  English  portion  of  an  entry  in  the  Vocabulary  File  is 
replaced  by  the  contents  of  the  change  card. 

(11)  Access  N  imber  Flip 

A  single  accession  number  may  be  removed  from  posting 
under  one  term  and  inserted  under  another  term  in  the  Posting 
Index  to  correct  mispostings. 

(12)  Delete  Generic 

This  program  deletes  single  entries  from  the  Generic  File. 

( 13)  Add  Generic 

This  program  adds  single  entries  to  the  Generic  File. 

(14)  Thesaurus  Update 

This  program  is  used  to  make  changes  to  the  Thesaurus 
File.  Change  cards  containing  code  numbers  and  designations 
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(e.g.,  GT,  PO,  SA,  RT,  and  "see"  terms)  are  read,  sorted, 
checked  for  validity  against  the  Vocabulary  File,  and  posted. 
Invalid  terms  are  printed  for  correction. 

(15)  Punch 

The  contents  of  the  Posting  Index  are  punched  into  cards 
for  transfer  to  the  IBM  7094  for  retrieval.  This  step  is  neces¬ 
sary  because  of  magnetic  tape  incompatibility  between  equip¬ 
ments. 

(16)  Compare 

This  progr  u.  compares  the  vocabulary  to  the  thesaurus  to 
find  terms  that  exist  in  the  vocabulary  but  do  not  appear  in  the 
thesaurus.  These  terms  and  invalid  terms  from  the  Thesaurus 
Update  are  reconciled  by  deleting  or  by  adding  in  the  proper 
place,  to  assure  one-to-one  correspondence  in  the  two  files. 

(17)  KWIC 

This  is  a  standard  program  provided  by  IBM  to  all  users. 

It  prints  a  permuted  title  listing  for  all  new  documents. 
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EQUIPMENT,  COSTS,  AND  EVALUATIONS 


EQUIPMENT 


NCR  304 


The  NCR  304  system  is  owned  by  the  University 
of  Dayton  and  is  used  primarily  for  nonscientific 
data  processing  problems.  In  addition  to  the 
following  equipment,  the  computer  also  has 
paper  tape  in  and  out  and  typewriter  control  at 
the  console. 


304- B 


with  4K  memory 


magnetic  tape  controller 


8  332 


magnetic  tape  handlers 


multipurpose  converter 
printer 


card  reader 


IBM  514 


IBM  7094-7044 


7094 


8  729 


card  punch 


The  7094-7044  system  has  been  rented  by  the 
Digital  Computations  Division,  a  service  facility 
for  all  of  W-PAFB,  for  about  one  year.  It  is 
used  primarily  for  scientific  computations,  and 
about  85  percent  of  the  programs  are  in  FORTRAN 
language. 

direct  couple  system  with  32K  memory 

Mod  IV  tapes 

disk  (10  million  words) 
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7044 

input-output  computer 

1402 

card  reader/punch 

6  1403 

line  printers 

2.  TIME  AND  COSTS 

The  University  of  Dayton  personnel  assigned  to  the  contract  con¬ 
sist  of  five  full-time  employees  (professional,  technical  for  subject  in¬ 
dexing)  at  the  University  and  five  full-time  employees  located  at  the 
Center. 

The  contract  with  the  University  of  Dayton  has  cost  the  Center 
approximately  $150,000  per  year  for  the  last  three  years.  This  in¬ 
cludes  system  development,  indexing,  retrieval  statement  composition, 
and  file  maintenance. 

The  search  program,  which  ran  approximately  30  minutes  on  the 
304,  takes  only  3  minutes  to  run  on  the  7094.  Some  400  searches  were 
made  during  the  past  year,  and  some  600  are  expected  this  year.  In 
addition,  10  to  15  manual  searches  are  made  each  month. 

Costs  on  the  7094  (at  $400  per  nour)  have  been  approximately 
$15,400  in  the  15  months  since  September  1964,  including  $2,300  in 
November  1965.  This  includes  the  cost  of  printing  KWIC  title  listings 
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and  operation  of  the  system.  Since  the  Center  is  not  charged  for  search 
time  on  the  Materials  Laboratory’s  IBM  7094,  they  feel  they  save 
$11, 000  by  using  this  computer. 

3.  EVALUATIONS 

The  information  retrieval  system  established  by  the  University  of 
Dayton  became  operable  in  mid- 1963.  At  that  time,  a  thesaurus  of  ap¬ 
proximately  10,000  terms  and  a  dictionary  containing  references  to  ap¬ 
proximately  6,500  documents  were  tabluated  on  the  NCR  304  computer, 
and  multiple  copies  of  each  were  forwarded  to  the  Laboratory.  It  was 
only  after  this  date  that  any  form  of  an  evaluation  could  be  made  on  the 
propriety  of  the  selected  system  and  its  various  components. 

The  use  of  links  and  roles  had  been  added  to  the  indexing  process 
in  order  to  reduce  the  number  of  false  drops  and  to  permit  highly  soph¬ 
isticated  searches.  It  became  desirable  to  determine  the  depth  of  in¬ 
dexing  being  performed  on  each  document  and  whether  links  and  roles 
were  serving  the  purpose  for  which  they  were  intended. 

(1)  Evaluation  of  Links  and  Roles  in  Searches 

A  study  of  the  value  of  links  and  roles  was  undertaken  by  a 
student  of  the  Air  Force  Institute  of  Technology  (AFIT)  as  his 
graduate  thesis.  Working  with  the  University  of  Dayton  staff,  he 


ran  actual  searches  on  which  to  base  his  evaluation.  In  this  study, 
roles  were  found  inferior  to  links.  The  use  of  links  showed  a  re¬ 
duction  of  irrelevant  information  by  over  56  percent  while  incur¬ 
ring  less  than  five  percent  loss  of  relevant  information.  The 
study  recommended  the  dropping  of  roles,  or  an  increase  of  the 
word  limit  for  computer  searches  from  40  to  100  (40-word  limit 
is  sufficient  when  using  links  alone).  The  recommendation  to 
drop  roles  was  accepted,  recognizing  that  it  was  almost  impos¬ 
sible  to  provide  definitions  which  would  assure  the  use  of  the 


same  role  in  a  question  as  in  the  original  indexing.  Complete 
results  of  this  study  are  given  in  ML  TDR  64-152  (see  Refer¬ 
ence  1). 


A  second  evaluation  of  links  and  roles  was  made  by  a  Uni¬ 
versity  of  Dayton  employee  experienced  in  setting  up  actual 
search  questions.  This  study  supplemented  the  first  and  con¬ 
firmed  the  clecision  to  drop  roles. 

(2)  Special  Indexing  Studies 

A  special  study  has  been  made  of  the  indexing  process. 
This  study  was  devoted  to  the  indexing  of  documents  in  which 
attention  was  paid  to  the  location  of  the  terms  indexed.  Each 
document  was  indexed  in  four  parts:  the  abstract,  summary  and 
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conclusions,  table  of  contents  and  figures,  and  the  body  of  the 
report.  The  indexing  was  done  in  the  indicated  sequence,  and  a 
color  code  was  used  to  distinguish  the  indexing  of  the  four  parts. 
The  indexing  of  each  subsequent  section  was  used  to  complement 
and/or  supplement  the  indexing  of  the  preceding  sections.  The 
results  of  this  study  are  tabulated  in  Tables  1  and  2  of  RTD-TDR- 
63-4263  (see  Reference  2). 

Realizing  that  different  disciplines  presented  distinct  in¬ 
dexing  problems,  the  documents  were  categorized  as  being  in  the 
area  of  ceramics,  chemistry,  metallurgy,  or  physics.  However, 
since  the  reports  did  not  necessarily  contain  all  four  sections 
from  which  terms  were  indexed,  i.  e.  ,  all  reports  did  not  have 
abstracts,  etc.  ,  additional  tabulations  were  made  of  those  re¬ 
ports  having  similar  sections.  In  addition,  the  results  of  this 
study  show  some  of  the  costs  added  to  the  system  by  the  use  of 
links  and  roles;  e.g.  ,  50  percent  more  entries  caused  by  use  of 
links  and  roles  undoubtedly  results  in  increased  indexer's  time, 
keypunch  expenses  for  time  and  materials,  additional  computer 
processing  time,  and  additional  postings. 
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(3)  Additional  Studies 


The  Center  has,  for  the  past  three  years,  run  searches  on 
projects  proposed  for  contract  and  distributed  these  to  the  proj¬ 
ect  offices  involved.  Following  these  searches,  a  questionnaire 
(see  Figure  6)  was  sent  to  each  recipient  to  determine  the  value 
of  the  particular  search.  At  the  end  of  1964,  some  42  replies  had 
been  received;  22  indicated  that  the  search  results  had  been  of 
help,  and  six  indicated  that  the  search  results  had  actually  can¬ 
celled  parts  of  projected  projects.  Some  183  searches  were  per¬ 
formed  this  past  year,  but  results  of  the  questionnaire  are  not  yet 
available. 


-31  - 


EVALUATION  OF  INFORMATION  RECEIVED  FROM 


________ Estimated 

“  Percent 

Material  vas  closely  related  to  my  vork  % 

Material  vas  moderately  related  to  my  vork  % 

Material  vas  remotely  related  to  my  vork  _ % 

Material  vas  totally  unrelated  to  my  vork  _ % 


Material  vas  nev  to  me 


Material  vas  not  nev  to  me 


Receiving  this  material  affected  the  course 
of  the  vork 


□ 


Receiving  this  material  affected  the  choice 
of  contractor 


I - 1 


This  material  indicated  some  anticipated  vork 
vas  unnecessary 

This  material  made  no  contribution  to  my  vork 


CD 

CD 


would  have  appreciated  _ _ _ 

Journal  Literature  CD  Patent  Literature  CD  _ 

s  material  adequately  pin-pointed?  YES  CD  NO  □ 

aments . 


FIGURE  6 

Search  Questionnaire 


CD 
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APPENDIX  B 


MAINTENANCE  OF  THE  THESAURUS 


THINGS  YOU  CAN  DO  TO  THE  VOCABULARY 


CODE  COMMENTS 

A  Add  a  NEW  word  to  the  system;  should  have  a 

number  which  has  never  been  used 

C  Change  the  spelling  of  an  existing  number.  Ef¬ 

fect  is  to  delete  one  word  and  add  another  with 
the  same  word  number 

D  Delete  existing  word  on  vocabulary  which  DOES 

NOT  have  postings  on  the  posting  index 

F  a.)  Flip  to  "zeros"  for  a  delete  only--those 

words  which  DO  HAVE  postings  on  the  posting 
index.  Effect  is  to  delete  the  word  from  the 
vocabulary  AND  the  postings  from  the  posting 
index;  new  word  is  NOT  added  to  vocabulary, 
b. )  Flip  to  another  word  number  for  delete 
and  add.  Effect  is  to  delete  one  word  from 
vocabularv  and  add  a  NEW  word  with  a  NEW 
number,  while  "flipping"  documents  from  the 
old  word  number  to  the  new  word  number,  de¬ 
leting  the  documents  from  the  old  word  number. 


THINGS  YOU  CAN  DO  TO  THE  THESAURUS 


COMMENTS 

Add  a  new  master  word  #  (not  previously  in  the  thesaurus)  using 
00  as  the  posting. 

Add  SA,  PO,  GT,  or  RT  to  an  already  existing  master  word  #  in 
the  thesaurus.  If  no  master  item  exists  (a  00  item)  then  program 
prints  error  and  new  card  is  not  added  into  thesaurus. 

Delete  master  word  #  and  ALL  corresponding  postings  using  00 
and  ERASE 
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Delete  individual  postings  SA,  PO,  GT,  or  RT  using  ERASE. 
This  is  used  when  master  word  item  is  to  remain  in  thesaurus 
with  or  without  other  postings. 


THINGS  YOU  CAN  DO  TO  THE  GENERIC  TAPE 


CODE  COMMENTS 

A  Add  a  NEW  PO  to  the  generic  tape;  both  the 

master  word  #  and  the  PO  word  #  MUST  be  in 
the  vocabulary 

D  Delete  an  existing  item  from  the  generic  tape; 

there  must  be  a  one-to-one  correspondence 
between  cards  and  items  you  wish  to  delete; 
i.  e. ,  one  card  must  be  punched  for  each  item 
you  wish  to  delete 


THINGS  YOU  CAN  DO  TO  THE  POSTING  INDEX 


Add  new  documents- -word  items  to  the  Posting  Index  (from  index 
cards  or  error  sheets).  Cards  will  be  matched  with  vocabulary 
(sorted  by  word)  and  output  will  be  matched  with  generic  tape. 

You  will  get  a  new  error  sheet,  also,  the  generic  tape  match 
output  will  be  merged  with  the  search  tape  and  a  supplement  will 
be  printed  (or  the  entire  search  tape,  if  desired). 

FLIPS- -the  flip  cards  from  the  vocabulary  update  will  be  used  to 
delete  access  #'s  and  add  them  under  new  master  word  #'s.  This 
output  WILL  be  matched  with  the  generic  tape,  which  output  in 
turn  will  be  merged  with  the  search  tape.  A  printout  can  be  made 
available  if  you  so  desire. 

"Bad"  access  numbers  can  be  deleted  from  the  posting  index. 

One  card  per  access  #  must  be  punched.  The  "bad"  documents 
will  be  deleted,  leaving  good  documents  untouched- -even  under 
the  same  master  word  #,  i.  e.  ,  you  can  delete  any  number  of 
"bad"  documents  under  a  certain  word  § ,  leaving  "good"  docu¬ 
ments  under  that  word  #.  Just  remember  that  one  card  must  be 
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punched  for  each  "bad"  document.  Also,  some  of  the  deleted 
documents  may  belong  elsewhere.  You  can  add  these  documents 
to  the  proper  word  simply  by  utilizing  this  feature  on  the  input 
cards.  If  additions  exist,  the  output  will  be  matched  with  the 
generic  tape  and  in  turn  will  be  merged  with  the  search  tape.  The 
effect  is  similar  to  that  of  a  flip  card- -the  difference  being  the 
ability  to  "flip"  certain  but  not  all  the  documents  under  a  given 
word  #. 


APPENDIX  C 


INDEXING  GROUND  RULES 
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INDEXING  GROUND  RULES 

The  elimination  of  role  indicators  simplified  the  indexing  pro¬ 
cedures  and  reduced  the  number  of  indexing  ground  rules.  Specific 
indexing  techniques  are  primarily  concerned  with  terminology  or  vocab¬ 
ulary  control  while  the  basic  indexing  procedures  remain  unchanged. 
First,  the  information  analyst  must  become  familiar  with  the  contents 
of  a  document  before  he  can  determine  its  important  concepts.  For  the 
majority  of  documents,  this  can  be  accomplished  by  reading  the  ab¬ 
stract  and  conclusions,  and  by  scanning  the  table  of  contents  and  the 
body  of  the  report.  The  key  terms  that  best  describe  the  report's 
contents  are  then  chosen,  and  links  are  used  to  maintain  separation  of 
the  terms  of  principal  concepts.  The  use  of  links  has  become  valuable 
in  separating  organic  compounds  and  preventing  the  "connection"  of 
substituent  groups  to  the  wrong  basic  structure. 

Any  indexed  terms  that  are  not  already  included  in  the  system's 
vocabulary  are  recorded  on  master  word  cards.  These  cards  are  re¬ 
viewed  once  a  week  by  all  staff  members.  This  method  permits  the 
revising  of  the  vocabulary  tape  prior  to  the  updating  of  the  Posting 
Index,  and  at  that  time,  only  keypunching  errors  should  be  rejected  by 


the  computer. 


The  changes  made  in  the  system's  vocabulary  made  it  advisable 
to  make  one  other  change  in  the  indexing  procedure.  Realizing  the  pos¬ 
sibility  that  users  might  ask  for  information  on  specific  materials,  a 
decision  was  made  to  use  more  scope  notes  on  the  index  cards  in  order 
to  simplify  the  screening  of  search  results.  Any  notes  that  an  informa¬ 
tion  analyst  thinks  would  be  beneficial  in  the  screening  process  are 
written  on  the  index  card  in  script  and  enclosed  in  parentheses.  These 
notes  are  not  keypunched  and,  therefore,  do  not  affect  the  computer 


programs. 
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SAMPLES  OF  OUTPUTS 


l*6C£C; 

1 45*003 

rh?.vJ3ICal  Properties 
pc  flO?  Properties 

Juesoii. 

-  ..?C„.  .^WL^AU!SOPEariES. 

SAMeci. 

RHOlilJ" 

64  f>50C 

PC  mETaL8 

refractory  metals 

43  am: 

PC  TRANSIT 10^  MCTAL8 

3V6?:;J:  * 

RH-jcTjm  ALLOTS 

3A0?rv: 

po  refractory  alloys 

?*:>3isc 

RH.mSJ'J-  A.nTI-OmIOE  .  . 

r-  logo*- 

PZ  A  TI^OMOES 

34  f»22*i ;  " 

BCRloF 

C-2AS.; 

PC  3  CRISES  ..  . 

34*300,. 

vicON*:  CCAT1NQS  ..  .... 

C; 73?OC  P%  COATING 

.  ....  ri..j*£IaLS  . _ _ _ 

?:  refractory  *etal  coatings 
3*»3:scc  pc  refractory  miTals 

3<:M50:  s»:  RfJODIU*i 

AT  .6^1:  03  transition  metals 

.2*l34S£  .  RH  nlt'M  BTi  trtn£ _ _ 

3«a:?DCr  PC  3IL1CIDES 

34*e:c:  ?H^CIJM*IP 

SAaSSO:  PC  refractory  alloys 

3<»£2C;-0  PC  RHOOIUM  ALLOYS 

'sa:*T5c:”“  "Th  cciu^Tcs  — 

3A2r5C“,  PC  REFRACTORY  ALLOYS 

?4;>20C:  PC  rhOOILm  ALLOYS 

?^'7CO;  RHcCUfi.PO 

24215 50 _ fia...  aEf-RACT.QRY.  ALLQIS _ 


3vb2COC 

PC 

RhOOIUM  alloys 

346500: 

“'HOC 

I0M*PT 

342350C 

PC 

REFRACTORY  ALLCY8 

34*20CM 

»C 

RhODIuh  ALLOYS 

'  3477*0:  ' 

fUM*RU 
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'h  SI  Mlif  R  Of  HH-FRF.NCF*  F"»a»  th*  t.  t  number  of 
ret  warn  ra  i  i»o.|  m  t|o>  n^ntl. 

*•  <  ON  TRA(  T  OR  t.RANI  Nt'UltFP  If  *pproprt*»*.  aniar 

»ha  apph'  »l't*  number  of  ihr  •  i«niv.n  i  or  gram  un'lrr  whirh 
»lir  M-pofi  waa  writ  I  an 

**A  ft  ,  M'f  PWO|rCT  NUMOFR  Enter  tl-  .ypprcpnate 
»■  Miinr,  department  *  dent  if ir rytmn.  «•»•  h  .ia  pr  •■*«  f  nitmhar, 
aiil-pr*»i*rt  number.  ayatem  number*.  tnsU  n  n*  bar.  air. 

«•  »  ORH.INA  TOP'S  REPORT  NUMPFR'^^  Tolar  i|w.  nffj. 

.  t.r  'rp  ft  i**init-»'t  by  whit'll  th#  ilorun.ant  will  li*  id*tififiad 
»t*t  .(IrM  I  ha  ortuinatiHg  atftviiy.  I  In  a  numhar  nniat 
I***  ••Oftj.ir  to  fbt*.  r».p.  .ft. 

•*»  MjiM  |V  in  I  H|ff  Nl'MHFR  If  iha  r*por*  hna  h**n 

I  "ii  *l**i  rqu  m  tntmhar*-  t t  #•»  tbr  *rtdmafor 
»f  *■%  th .«  .f  ti  .  •»!,  ,i|ao  aniar  ihia  rnimbariai. 

M.  AVAII  API!  II  Y  I. IMITATION  NO  IK  TS  Fn»*r  any  Itm* 
*^*i  it :  ..«t  luriht  *  dtaaamtnai i*n  *f  H»a  rap.trf,  othar  than  thoaal 


impnaarl  t*v  aariirlty  *  laaatftr  nlmn,  uaitty  arattdard  afa|#m*nta 
aut  h  aa- 

fl)  **<^uallflad  raquaatara  nut v  obtain  >  opiaa  ,%f  thia 
rapnrt  frori  DOf." 

*  )\  armoun*  «*i«*mI  itml  tliaaatfttn.H inn  **f  thia 

raport  Ity  PDf*  ia  tt»*l  nnlh»rt  rati.  *' 

t  U  **ll.  S.  f«nv*ttim*t*t  a»f**triaa  may  oht«m  rnpiaa  of 
Ihia  raj*ort  dir**tly  frt.at  |)|»T.  nth  at  •|naltti*«f  OPT 
naara  ahall  raquaat  thmn(h 

t* 

ill  **ll  S  mil  it  mi  v  again  i*a  may  obtain  ropiaa  of  thia 
rap»»rt  d«r#rtlv  from  Pf>C.  Othar  qnalift ad  naara 
ahall  ra»pt*ai  (lit •••it'll 

*• 

i*i  *# Alt  diatr iltntion  »*|  Ihia  rapnn  ia  rniiiMlIal  <>iai- 
iflad  PIM*  naot*.  •  hall  raipiaat  Ihtouyh 

M 

ll  thr  rap*. it  hna  !•****  tnrniah**l  t**  th*  Oft*,  a  of  T*«  hniral 
Sat  v»*  a*  .  I  *np:ittni*nt  "I  »  •.•*.«**rf,  #•,  f* .«  aal«*  i»*  th*  pidtlt*.  intli* 

•  i*r  thia  I**  »  and  *t»l*»  Hu*  put  **.  if  ltm«wtk 

11.  31  (l*f 'I N  t  AR\  V  » |  |*  *»■  Hat*  f»tr  M«f«fti mttal  **plana> 

ior\  f.iaa. 

I.’.  SfONSOWIN..  hll.1l  AkV  A<  TIVU  Y:  1'ninll*M*Ml 

tl*a  art m**itat  p*«»i*»  I  ntfia*  nr  lahttratory  apnnanrtng  tpai* 
i»i«t  I  t)  I  ha  i*a*an  h  and  'lavalopnirnt.  Ini  -hula  addraaa. 

I  1  AUSTPAOT*.  F.ntar  nn  ahalra*  I  giving  a  hriaf  and  fa«  Inal 
*.um •*'.••! y  of  Hia  dn«  umant  itnlirativa  of  th**  raport.  avan  thnugh 
i*  it* >* v  alat«  appaar  a|aawh*ra  in  Iha  t.**dy  of  Hta  ta*  hniral  r* 
putt  If  additional  apa*  *  in  r*«imr*d.  »  ,  "r.imuMii  'o  *-hm|  abaM 
i»*  artat  |*at| 

It  »a  highly  daatrahlr  that  th*  abatra*  t  «»f  <  laaaifiad  raport  a 
l.a  nm  laaaifiad  F.arh  paragraph  *»f  iha  abairat  t  ahall  and  wtih 
an  mtliratfon  of  tha  military  aarnntv  rlaaatfit  aimn  ..f  tha  m 
!•  •final ton  m  th*  paragraph  rapr*a*nt*d  aa  t  T t »  Hi  in  nrr l*y 

Thara  la  nn  limitation  «*n  th*  langih  «d  iha  ahatta*  t  ||ow 
*v*t  th*  augg*atad  langth  ia  fmm  I  AO  to  word* 

M  KF.Y  WORDS:  K*y  wn.rda  ara  ta.  hmt  alty  maaningful  term* 
•>r  abort  phraaaa  tlmi  t  harat  lartra  a  raport  and  mav  h*  naad  aa 
in.|i  V  antrt*a  f*»r  cataloging  tha  rap«»tt  K*v  Worda  m*iat  h* 
s*tr  tad  an  that  no  aa>  unty  .  laaatfit  ation  ia  rrtpntad  Idanti* 
fiara  an.  h  aa  a*|nipm*nt  modal  daatgnation.  Had*  nam*  military 
profa.  i  i-.ida  nama.  gaographit  hu  atU'n.  may  1^*  na**f  aa  h*y 
v -*rda  hut  will  ha  followad  hy  an  intli.  almn  nf  larhni.  al  rr>n 
tr*t  Th*  aaa»gnm*t*t  *»f  l.nha  ml*a  and  w*ighta  ta  optional 


S#rtm.v  CU*«>fir»tto*» 


